{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Saving Data Locally \n", "\n", "Grid Status provides standardized access to the APIs and data provided by independent system operators (ISOs), but it does not distribute the data itself.\n", "\n", "If you want to save the data locally, you can use the `save_to` parameter to make it easier to save and access the data in the future. This way, you won't have to redownload the data every time you want to use it. \n", "\n", "You can then use the `load_folder` method to reload the saved data." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "import gridstatus\n", "import plotly.express as px" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "iso = gridstatus.NYISO()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, let's use Grid Status to grab a few years of fuel mix data from NYISO. We can specify the local directory where we want the data to be saved using the save_to parameter. This parameter is supported by all methods in Grid Status, so you can use it with any method that retrieves data." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 35/35 [00:51<00:00, 1.49s/it]\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TimeDual FuelHydroNatural GasNuclearOther Fossil FuelsOther RenewablesWind
02020-01-01 00:05:00-05:002350.03279.01883.05418.01.0277.0883.0
12020-01-01 00:10:00-05:002207.03113.01801.05420.01.0273.0851.0
22020-01-01 00:15:00-05:002133.03018.01848.05422.01.0275.0852.0
32020-01-01 00:20:00-05:002047.03059.01929.05421.01.0274.0863.0
42020-01-01 00:25:00-05:002035.03092.01946.05419.01.0271.0861.0
...........................
3129142022-11-30 23:40:00-05:002454.02434.03673.03342.00.0253.01651.0
3129152022-11-30 23:45:00-05:002447.02352.03628.03342.00.0252.01684.0
3129162022-11-30 23:50:00-05:002420.02309.03566.03342.00.0253.01715.0
3129172022-11-30 23:55:00-05:002342.02256.03538.03342.00.0252.01728.0
3129182022-12-01 00:00:00-05:002253.02283.03557.03342.00.0251.01673.0
\n", "

312919 rows × 8 columns

\n", "
" ], "text/plain": [ " Time Dual Fuel Hydro Natural Gas Nuclear \\\n", "0 2020-01-01 00:05:00-05:00 2350.0 3279.0 1883.0 5418.0 \n", "1 2020-01-01 00:10:00-05:00 2207.0 3113.0 1801.0 5420.0 \n", "2 2020-01-01 00:15:00-05:00 2133.0 3018.0 1848.0 5422.0 \n", "3 2020-01-01 00:20:00-05:00 2047.0 3059.0 1929.0 5421.0 \n", "4 2020-01-01 00:25:00-05:00 2035.0 3092.0 1946.0 5419.0 \n", "... ... ... ... ... ... \n", "312914 2022-11-30 23:40:00-05:00 2454.0 2434.0 3673.0 3342.0 \n", "312915 2022-11-30 23:45:00-05:00 2447.0 2352.0 3628.0 3342.0 \n", "312916 2022-11-30 23:50:00-05:00 2420.0 2309.0 3566.0 3342.0 \n", "312917 2022-11-30 23:55:00-05:00 2342.0 2256.0 3538.0 3342.0 \n", "312918 2022-12-01 00:00:00-05:00 2253.0 2283.0 3557.0 3342.0 \n", "\n", " Other Fossil Fuels Other Renewables Wind \n", "0 1.0 277.0 883.0 \n", "1 1.0 273.0 851.0 \n", "2 1.0 275.0 852.0 \n", "3 1.0 274.0 863.0 \n", "4 1.0 271.0 861.0 \n", "... ... ... ... \n", "312914 0.0 253.0 1651.0 \n", "312915 0.0 252.0 1684.0 \n", "312916 0.0 253.0 1715.0 \n", "312917 0.0 252.0 1728.0 \n", "312918 0.0 251.0 1673.0 \n", "\n", "[312919 rows x 8 columns]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "iso.get_fuel_mix(start=\"Jan 1, 2020\", end=\"Dec 1, 2022\", save_to=\"nyiso_fuel_mix/\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This will retrieve the fuel mix data for the specified time period and save it to the specified local directory. You can then use the `load_folder` method to reload the saved data.\n", "\n", "Before we do that, let's look at our load folder. As you can see below there are now multiple files\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/Users/kanter/.zshenv:1: unknown file attribute: y\n", "NYISO_get_fuel_mix_20200101_20200201.csv\n", "NYISO_get_fuel_mix_20200201_20200301.csv\n", "NYISO_get_fuel_mix_20200301_20200401.csv\n", "NYISO_get_fuel_mix_20200401_20200501.csv\n", "NYISO_get_fuel_mix_20200501_20200601.csv\n", "NYISO_get_fuel_mix_20200601_20200701.csv\n", "NYISO_get_fuel_mix_20200701_20200801.csv\n", "NYISO_get_fuel_mix_20200801_20200901.csv\n", "NYISO_get_fuel_mix_20200901_20201001.csv\n", "NYISO_get_fuel_mix_20201001_20201101.csv\n" ] } ], "source": [ "!ls nyiso_fuel_mix/ | head -n 10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's load the data back in. By default, the data will be loaded with UTC time. We can use the time_zone parameter to specify a different timezone. In this case, let's use the default time zone for NYISO" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 35/35 [00:00<00:00, 133.26it/s]\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TimeDual FuelHydroNatural GasNuclearOther Fossil FuelsOther RenewablesWind
02020-01-01 00:05:00-05:002350.03279.01883.05418.01.0277.0883.0
12020-01-01 00:10:00-05:002207.03113.01801.05420.01.0273.0851.0
22020-01-01 00:15:00-05:002133.03018.01848.05422.01.0275.0852.0
32020-01-01 00:20:00-05:002047.03059.01929.05421.01.0274.0863.0
42020-01-01 00:25:00-05:002035.03092.01946.05419.01.0271.0861.0
...........................
3129142022-11-30 23:40:00-05:002454.02434.03673.03342.00.0253.01651.0
3129152022-11-30 23:45:00-05:002447.02352.03628.03342.00.0252.01684.0
3129162022-11-30 23:50:00-05:002420.02309.03566.03342.00.0253.01715.0
3129172022-11-30 23:55:00-05:002342.02256.03538.03342.00.0252.01728.0
3129182022-12-01 00:00:00-05:002253.02283.03557.03342.00.0251.01673.0
\n", "

312919 rows × 8 columns

\n", "
" ], "text/plain": [ " Time Dual Fuel Hydro Natural Gas Nuclear \\\n", "0 2020-01-01 00:05:00-05:00 2350.0 3279.0 1883.0 5418.0 \n", "1 2020-01-01 00:10:00-05:00 2207.0 3113.0 1801.0 5420.0 \n", "2 2020-01-01 00:15:00-05:00 2133.0 3018.0 1848.0 5422.0 \n", "3 2020-01-01 00:20:00-05:00 2047.0 3059.0 1929.0 5421.0 \n", "4 2020-01-01 00:25:00-05:00 2035.0 3092.0 1946.0 5419.0 \n", "... ... ... ... ... ... \n", "312914 2022-11-30 23:40:00-05:00 2454.0 2434.0 3673.0 3342.0 \n", "312915 2022-11-30 23:45:00-05:00 2447.0 2352.0 3628.0 3342.0 \n", "312916 2022-11-30 23:50:00-05:00 2420.0 2309.0 3566.0 3342.0 \n", "312917 2022-11-30 23:55:00-05:00 2342.0 2256.0 3538.0 3342.0 \n", "312918 2022-12-01 00:00:00-05:00 2253.0 2283.0 3557.0 3342.0 \n", "\n", " Other Fossil Fuels Other Renewables Wind \n", "0 1.0 277.0 883.0 \n", "1 1.0 273.0 851.0 \n", "2 1.0 275.0 852.0 \n", "3 1.0 274.0 863.0 \n", "4 1.0 271.0 861.0 \n", "... ... ... ... \n", "312914 0.0 253.0 1651.0 \n", "312915 0.0 252.0 1684.0 \n", "312916 0.0 253.0 1715.0 \n", "312917 0.0 252.0 1728.0 \n", "312918 0.0 251.0 1673.0 \n", "\n", "[312919 rows x 8 columns]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = gridstatus.load_folder(\"nyiso_fuel_mix/\", time_zone=gridstatus.NYISO.default_timezone)\n", "df" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To see that the data is correct, we can plot the data. Let's plotly the average hourly fuel mix" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": "0510152001000200030004000variableDual FuelHydroNatural GasNuclearOther Fossil FuelsOther RenewablesWindNYISO Average Hourly Fuel MixTimeProduction (MWh)" }, "metadata": {}, "output_type": "display_data" } ], "source": [ "hourly = df.set_index(\"Time\").resample(\"H\").mean()\n", "avg_hourly = hourly.groupby(hourly.index.hour).mean()\n", "\n", "fig = px.line(avg_hourly, y=avg_hourly.columns, title=\"NYISO Average Hourly Fuel Mix\")\n", "fig.update_yaxes(title=\"Production (MWh)\")\n", "fig.show(\"svg\", width=1200, height=600)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3.10.2 64-bit ('isodata')", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.2" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "49f14642123d0cc1afa9fa45716ed5f1e915189c28b01efe02a8b7ec3c0a3fce" } } }, "nbformat": 4, "nbformat_minor": 2 }